Content Management System using Sources of Experience Data and Modules for Quantification and Visualization

ABSTRACT

A semantic note taking system and method for collecting information, enriching the information, and binding the information to services is provided. User-created notes are enriched with labels, context traits, and relevant data to minimize friction in the note-taking process. In other words, the present invention is directed to collecting unscripted data, adding more meaning and use out of the data, and binding the data to services. Mutable and late-binding to services is also provided to allow private thoughts to be published to a myriad of different applications and services in a manner compatible with how thoughts are processed in the brain. User interfaces and semantic skins are also provided to derive meaning out of notes without requiring a great deal of user input.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/462,816 filed May 3, 2012. This application claims priority from U.S.Provisional Patent Application 61/518,322 filed May 3, 2011.

This application claims priority from U.S. Provisional PatentApplication 61/518,324 filed May 3, 2011.

This application is a continuation-in-part of U.S. patent applicationSer. No. 12/924,094 filed Sep. 20, 2010. U.S. patent application Ser.No. 12/924,094 filed Sep. 20, 2010 claims priority from U.S. ProvisionalPatent Application 61/276,975 filed Sep. 18, 2009. U.S. patentapplication Ser. No. 12/924,094 filed Sep. 20, 2010 is acontinuation-in-part of U.S. patent application Ser. No. 12/322,046filed Jan. 27, 2009, Ser. No. 12/381,594 filed Mar. 13, 2009, and Ser.No. 12/802,740 filed Jun. 11, 2010.

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/345,023 filed Jan. 6, 2012. U.S. patent application Ser. No.13/345,023 filed Jan. 6, 2012 claims priority from U.S. ProvisionalPatent Application 61/460,813 filed Jan. 7, 2011. U.S. patentapplication Ser. No. 13/345,023 filed Jan. 6, 2012 is acontinuation-in-part of U.S. patent application Ser. No. 12/322,046filed Jan. 27, 2009.

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/349,733 filed Jan. 13, 2012. U.S. patent application Ser.No. 13/349,733 filed Jan. 13, 2012 claims priority from U.S. ProvisionalPatent Application 61/461,120 filed Jan. 13, 2011. U.S. patentapplication Ser. No. 13/349,733 filed Jan. 13, 2012 is acontinuation-in-part of U.S. patent application Ser. No. 12/322,046filed Jan. 27, 2009.

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/349,901 filed. Jan. 13, 2012. U.S. patent application Ser.No. 13/349,901 filed Jan. 13, 2001 claims priority from U.S. ProvisionalPatent Application 61/461,097 filed Jan. 13, 2011. U.S. patentapplication Ser. No. 13/349,901 filed Jan. 13, 2011 is acontinuation-in-part of U.S. patent application Ser. No. 12/381,594filed Mar. 13, 2009. U.S. patent application Ser. No. 12/381,594 filedMar. 13, 2009 is a continuation-in-part of U.S. patent application Ser.No. 12/322,046 filed Jan. 27, 2009.

All applications listed in this section “Cross Reference to RelatedApplications” are hereby incorporated by reference to this applicationin their entirety.

FIELD OF THE INVENTION

The invention relates generally to online social media. Moreparticularly, the present invention relates to collecting and enrichinginformation in a network environment.

BACKGROUND

Today, a tremendous number of online services and applications areavailable. The existing online services are related to virtually everyaspect of life, including work, relationships, health, entertainment,news, etc. Access to these online services typically only requires anInternet connection. However, the abundance of services also causesinformation overload. As a result, disorganized information is floatingaround many users lives and brains. Instead of positively influencingour lives, the services become cumbersome and are often neglected. Inparticular, the advent of social media has made it easier to shareinformation with more people and make connections with people who weotherwise would not. However, the ubiquity of social media (such asthrough social networks) has created a further explosion in theinformation content of our lives and relationships. This informationclutter reduces a user's productivity and shortens the user's attentionspan.

Typically, ideas begin with private thoughts that are later refined andshared to friends and, perhaps, the public at large. Who these ideas areto be shared with often evolve over time. Existing online services arenot amenable to this process of developing ideas and deciding where toshare them at a later time. For example, existing social media forcesideas immediately into the public domain. Information or media posted onsocial networks (e.g. Facebook.com, Twitter.com, and Flickr.com) orblogs (e.g. Wordpress.com and Blogger.com) are immediately accessible byother users. To post information in existing social media requiresupfront cognitive decision-making related to the audience level of theinformation at the time the information was collected or generated. Thisprocess does not match the way human brains function.

Systems, such as emailing to oneself and note-taking applications (e.g.Evernote.com) exist for users to store private thoughts. However, theseexisting systems are not integrated with other services or applications.In other words, a user would have to access and sign on to multipleapplications simultaneously to utilize information stored in theapplication storing the private thoughts. Because of the lack ofintegration and inconvenience of these systems, they are seldom used,therefore, ideas are often forgotten. In addition, existing note-takingapplications require users to provide all of the information content tothe note. By forcing a user to provide all of the information content,the note-taking process is typically tedious, error-prone, andincomplete.

The present invention addresses at least the above-described difficultproblems and advances the art with a semantic note taking system andmethod.

SUMMARY OF THE INVENTION

The present invention is directed to a semantic note taking system andmethod for collecting information, enriching the information, andbinding the information to services. A plurality of users arecommunicatively connected to an application server to create one or morenotes that can be bound to one or more of a plurality of services andstored in a database. The application server operates a plurality offunctions, including a note taking function for allowing each of theusers to create one or more notes, a categorizing function for labelingeach of the notes with one or more changeable categories, a contextfunction for associating one or more context traits with each of thenotes, a binding function to establish one or more changeable bindingrules for each of the notes, wherein the binding rules determine one ormore of the services where the note is to be bound, and wherein thebinding rules are related to the content of the note, the categories ofthe note, the context traits of the note, a user binding selection, orany combination thereof, and a communication function for communicatingeach of the notes to one or more of the services where the note is to bebound.

A note includes text, audio media, visual media, audio-visual media,recorded data, a weblink, a pointer to an information source, or anycombination thereof. In an embodiment, one or more of the context traitsassociated with the note includes a time, a location, physical datarelating to the user device used to generate the note, or anycombination thereof. In another embodiment, relevant data is derivedbased on one or more of the context traits and the relevant data isattached to the note. The relevant data can be derived by matching oneor more of the context traits with data from an information module suchas a calendar, an address book, a contact list, a user profile, a userhistory, or any combination thereof. In an embodiment, one or more ofthe context traits associated with the note is based on one or morecategories of the note, the content of the note, a user history, or anycombination thereof.

In a preferred embodiment, the binding rules include one or morepublication properties, such as an audience level for the note and/orone or more locations to publish the note. The locations to publish thenote can include a private domain of the user, one or more friends ofthe user, one or more computer-implemented social networks, a blog, anonline discussion board, a website, or any combination thereof. In anembodiment, the binding rules are automatically determined based atleast partially on one or more of the categories, one or more contexttraits, the note content, a user history, or any combination thereof.

The application server of an embodiment of the present inventionoperates a suggestion function for suggesting one or more services,binding rules, and/or categories. The suggestions can be based at leastpartially on one or more the categories, one or more of the contexttraits, the note content, a user history, one or more binding rules, orany combination thereof. In an embodiment, a note is automatically houndto one of the suggested services. Preferably, the suggestion functionuses a user history of selections of services and/or categories.

In an embodiment of the present invention, a user interface is providedto allow user entry of the categories and binding rules for a note,wherein the user interface includes a first symbol associated with thecategories of the note and a second symbol associated with the bindingrules of the note. Preferably, the user entry of the first symboltriggers a display of one or more of the categories suggested to theuser and selectable by the user. Similarly, the user entry of the secondsymbol triggers a display of one or more of the binding rules suggestedto the user and selectable by the user. In another embodiment, a userinterface corresponding to each of the services where a note can bebound is provided. Each of the service-specific user interfaces includesone or more buttons for initiating an action for the servicecorresponding to the user interface. Preferably, a single click of oneof the buttons initiates creation of the note, automatic addition ofcontent to the note, automatic labeling of the note with categories, andcommunication of the note to the service corresponding to the userinterface.

In an embodiment, the user device operated by users to access theapplication server is a mobile device that communicates with theapplication server over a wireless network. The services where a notecan be bound are selected from the group consisting of a website, a webapplication, a computer-implemented social network, a blog, a reviewwebsite, a product review website, an entertainment website, a healthapplication, a medical application, an online retailer, an emailapplication, a research application, a clinical application, a calendar,and an address book.

In another embodiment a method for collecting information and enrichinginformation. A note taking function is provided for allowing a user tocreate a note on a user device. The created note can then be storedeither on the user device or remotely via an application server.Examples of notes are text, audio media, visual media, audio-visualmedia, recorded data, a weblink, a pointer to an information source, orany combination thereof.

An identification function is used to identify or capture the user'sinterests and needs from a plurality of sources of experience data.Examples of sources of experience data are one or more social networks,one or more financial services, one or more note-taking services, one ormore location check-in apps, one or more email programs, one or morephoto-sharing services, one or more calendars, or any combinationthereof. It is noted that the user is registered which these sources(i.e. linked accounts or services), which allows the function to gainaccess to the data pertaining interests and needs of the user.

A template function is used for defining one or more templates fororganizing data capture and visualizing the organized data. Thetemplates can then be stored either on the user device or remotely viaan application server. Templates can also be automated based on context.Templates can also be a suggested list of templates.

A pre-selection function is used for selecting one or more of the storedtemplates based on the identified user interests and needs and based onthe created note. The user can manually select a template from a list ora list of suggested templates. Templates can also be suggested to theuser in an automated fashion.

In one aspect, a feedback function could be used for providing theenriched note to said user. This function performs based on thepre-selected template a quantitative analysis and a visualization of thequantitative analysis to the user of the enriched note.

Similarly to this method, a system for collection information andenriching the information is provided. An application server is used tocommunicatively connect a plurality of users to the application serverto create one or more notes. A plurality of services are communicativelyconnected to the application server, and the notes can be bound to oneor more of these services. A database is used for storing the notescreated by said users. The application server operates a plurality offunctions such as (described supra) the note taking function, theidentification function, the template function, the pre-selectionfunction, and the feedback function.

BRIEF DESCRIPTION OF THE FIGURES

The present invention together with its objectives and advantages willbe understood by reading the following description in conjunction withthe drawings, in which:

FIG. 1 shows an example of a semantic note taking system according to anembodiment of the present invention.

FIG. 2 shows an example of mutable binding of notes to multiple servicesaccording to an embodiment of the present invention.

FIG. 3 shows a flow chart of an example note creation, enrichment, andbinding according to an embodiment of the present invention.

FIG. 4 shows a flow chart of example steps taken by a user according toan embodiment of the present invention n.

FIG. 5 shows an example of a suggestion function according to thepresent invention.

FIG. 6 shows an example user interface for semantic note takingaccording to an embodiment of the present invention.

FIG. 7A and FIG. 7B show examples of semantic skin interfacescorresponding to services according to an embodiment of the presentinvention.

FIG. 8 shows the core services of an exemplary embodiment of the presentinvention.

FIG. 9 shows the technology services underlying the core services shownin FIG. 8.

FIG. 10 shows a flow chart of an example login and authenticationfunction in an exemplary embodiment of the present invention.

FIG. 11 shows a flow chart of an example Queue service according to anexemplary embodiment of the present invention.

FIG. 12A and FIG. 12B shows an example of using semantic note takingwith online retailers according to an embodiment of the presentinvention.

FIG. 13A and FIG. 13B show an example of generating a note summary andprivate URL from a note, encrypting the note summary and private URL toform encoded text, generating a two-dimensional QR code from the encodedtext, and accessing the note by decrypting the encoded text to identifythe private URL according to an embodiment of the present invention.

FIG. 14 shows an example of linking a semantic note with a physicalobject through a two-dimensional bar code according to an embodiment ofthe present invention.

FIG. 15 shows an example of an outline of a structure with experiencedata sources annotating behavioral progress application according to anembodiment of the present invention.

FIG. 16A, FIG. 16B, and FIG. 16C show examples of Catch.com functionsand annotated lifestream according to an embodiment of the presentinvention.

FIG. 17A, FIG. 17B, FIG. 17C, and FIG. 17D show examples ofcontext-aware modules for tracking, insights and coaching feedbackaccording to an embodiment of the present invention.

FIG. 18A, FIG. 18B, and FIG. 18C show examples of providing positivepsychology and feedback to the user according to an embodiment of thepresent invention.

FIG. 19 shows an example of a Health tracker according to an embodimentof the present invention.

FIG. 20A, FIG. 20B, FIG. 20C, and FIG. 20D show examples of an expensetracker according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The abundance of online services provides users with opportunities toperform many tasks and connect with numerous people from nearly anylocation. However, the large number of services often cause informationoverload and a user cannot efficiently organize the explosion ofinformation content concerning the user's life, work, and relationships.The present invention is directed to a semantic note taking system andmethod to collect, enrich, and bind information to services. Semanticnote taking allows users to effortless offload information from privatethought streams for later determination of where to bind these thoughts.In addition, semantic note taking minimizes the friction to record andshare ideas by automatically enriching the note and reducing unnecessaryuser input. In other words, the present invention is directed to asystem and method to collect unscripted data, add more meaning and useout of the data, and bind the data to services.

FIG. 1 shows a preferred embodiment of a semantic note taking system 100according to the present invention. The semantic note taking system 100includes an application server 110 that is accessible by a plurality ofusers for creating one or more notes, categorizing the notes, andbinding the notes to one or more services S₁-S_(M). The users U₁-U_(N)connect to the application server 110 through any device capable ofbeing communicatively connected to a network, such as the Internet. Theuser devices can include a computer, a laptop, a personal digitalassistant (PDA), a cell phone, a mobile device, or a smart phone. FIG. 1shows user U₁ with a mobile phone 140 that is connected to theapplication server 110, such as through a wireless network.

A note taking function, operated by the application server 110, allowseach of the users U₁-U_(N) to create one or more notes. Notes caninclude text, audio media, visual media, audio-visual media, recordeddata, a weblink, a pointer to an information source (e.g. a bookmark),or any other type of data. Notes created by the users U₁-U_(N) aretransmitted from the user devices to the application server 110 andstored in the database 120 for later binding to one or more of theservices S₁-S_(M).

The application server 110 also operates a categorizing function tolabel the notes with one or more categories. In an embodiment, thecategorizing function allows users U₁-U_(N) to label the notes, thoughautomatic or suggested note labeling is also possible. It is noted thatone or more of the categories of each note is changeable. Labeling anote with categories enriches the note. The note is further enriched bythe context function operated by the application server 110, whichassociates one or more context traits with the note. Context traitsassociated with a note can be based on one or more of the categories ofthe note, the content of the note, a user history, or any combinationthereof. Examples of context traits include a time, a location, andphysical data relating to the user device. For example, the mobile phone140 of user U₁ is capable of measuring the location where the note wascreated (e.g. by using GPS measurements) and this location can beautomatically associated with the note without user input, therebyenriching the note.

Context information is not restricted to a time-stamp or location-stampassociated with the note. In a preferred embodiment of the presentinvention, relevant data is derived based on one or more of the contexttraits. Relevant data can be derived by matching one or more of thecontext traits with data from an information module 130, such as acalendar, an address book, a contact list, a user profile, a userhistory, or any combination thereof. For example, a time context traitwhen the note is created is matched with a meeting time indicated by acalendar information module. Because the calendar includes relevantdata, e.g. the participants of the meeting and the discussion topics ofthe meeting, this relevant data can be attached to the note to furtherenrich the note.

A binding function is provided to establish one or more binding rulesfor a note. The binding rules determine one or more services S₁-S_(M) orapplications where the note is to be bound. The binding rules arerelated to the content of the note, the categories of the note, thecontext traits of the note, a user binding selection, or any combinationthereof. In an embodiment, the binding rules of a note are automaticallydetermined based at least partially on one or more categories of thenote, one or more context traits, the content of the note, the userhistory, or any combination thereof.

The services S₁-S_(M) where a note can be bound can include a website, aweb application, a computer-implemented social network, a blog, a reviewwebsite, a product review website, an entertainment website, a healthapplication, a medical application, an online retailer, an emailapplication, a research application, a clinical application, a calendar,an address book, or any combination thereof. A note that is bound to aservice is communicated to the service. What is done with the note afterit is communicated depends on the particular functions of the serviceand the content of the note. For example, a note bound to a blog couldbe published on the blog and a note bound to a research application caninclude data for analysis by the research application. In addition tocommunicating a note to a service, in an embodiment of the presentinvention, data can be extracted from the service where the note isbound and the extracted data is added to the note.

The services S₁-S_(M) shown in FIG. 1 are generally not integrated andsome of the services S₁-S_(M) where a note can be bound may also requireuser login or authentication. To facilitate the integration of theservices and allow access to the services, in an embodiment, theapplication server 110 includes a login function to provide logininformation for access to the services. Integration of the servicesenables a user to collect or create information content and offload thecontent to a single location, instead of having to interact with amyriad of different services separately.

In a preferred embodiment of the present invention, the binding rulescomprise one or more publication properties. The publication propertiesinclude an audience or privacy level for the note and the location topublish the note. FIG. 2 shows three notes N₁, N₂ and N₃ created by userU₁ and the binding of these notes to different services. The audiencelevels generally include private, social, and public designations. Thelocations to publish the note includes a private domain of the user, oneor more friends F₁-F₃ of the user, one or more computer-implementedsocial networks, a blog, an online discussion board, a website, or anycombination thereof. FIG. 2 shows note N₁ bound 210 to a private journaland bound 220 to an email application. In other words, the binding rulesfor note N₁ indicate that the note should be communicated to the journalservice and the email application with the appropriate audience levels.

A note can be bound to any number of services at any time. Importantly,the binding of the note is changeable. In other words, a user can changethe binding rules to alter the location where a note is to be bound. InFIG. 2, note N₁ is originally bound 210 to a journal but is later bound230 to Social Network 1. Similarly, note N₃ is originally bound 240 to ablog, hut is later bound 250 to an electronic commerce website. Themutable binding rules and late-binding method of the present inventionis in accord with how ideas are typically generated in our brains. Inparticular, ideas and information typically start as private thoughtsthat are later shared and discussed with people we know. Finally, theinformation is published to the world. This is in contrast to existingsocial media where generated media are forced immediately into thepublic realm.

FIG. 2 also shows note N₁ published and communicated to associates orfriends F₁-F₃ of user U₁. By publishing note N₁ to friends F₁-F₃, an adhoc community or an ad hoc social network is developed around note N₁.In an embodiment, one or more of the friends F₁-F₃ can change the noteN₁. In particular, data can be extracted from one of the friends F₁-F₃and the extracted data is added to the note N₁.

FIG. 3 and FIG. 4 show flow charts for exemplary methods of creating,enriching, and binding the notes according to the present invention. Thesteps circled with a dashed line 310 in FIG. 3 show the enrichment ofthe notes by labeling the note with categories, associating contexttraits with the note, and deriving relevant data based on the contexttraits. FIG. 4 shows the steps undertaken by a user of the semantic notetaking system of the present invention.

FIG. 3 and FIG. 4 also include the steps of suggesting binding rules andreceiving suggestions, respectively. The application server operates asuggestion function for suggesting one or more services where the notecan be bound to the user who created the note. FIG. 5 shows the inputsfor the suggestion function 510, which can include the content of thenote, the categories of the note, the context traits, and/or the userhistory. Based on these inputs, the suggestion function 510 suggestsbinding rules 520, such as a list of suggested services where the notecan be bound. For example, FIG. 5 shows a ranked list of services with“Service 5” being the top-ranked service where the note can be bound.

In an embodiment, the suggested services or binding rules are selectableby the user, where the user selection binds the note to the selectedservice. User selections can be stored for reference by the suggestionfunction. For example, a first note is created by a user, labeled with“Category 4, and bound to “Service 5” by user selection. Because of thisuser selection, “Service 5” would be suggested for a second note alsolabeled with “Category 4”. It is noted that the first and second notescan be created by the same user, or they can be created by differentusers. In other words, the suggestion function 510 can rely on the userhistory of the same user or the user history of a community of users. Itis noted that the association of a service to a category is mutable andevolvable depending on user behavior. For example, though “Category 4”and “Service 5” are associated in the above example, the association ofservices for “Category 4” could change to “Service 3” if usersfrequently alter the selection from “Service 5” to “Service 3” for noteshaving a label of “Category 4”.

In an embodiment, a note is automatically bound to one of the suggestedservices without user selection. In a preferred embodiment, the storageof past user selections enables intelligent and automatic binding ofnotes to services. Automatic binding of notes to services reduces theneed for user input.

The suggestion function 510 is also capable of suggesting categories tolabel a note in addition to or replacement of suggesting binding rules.Suggestions of categories can be analogous to the suggestions ofservices described in the previous paragraphs. In particular,suggestions of categories can be based on the content of the note, thecontext traits of the note, other categories of the note, a userhistory, the binding rules established for the note, the servicesdetermined by the binding rules, or any combination thereof. Forexample, a note contains a description of a product. Based on this notecontent, a category named “product” is suggested to the user forlabeling the note. In another example, the association of a service anda category based on a user history can be used to suggest one or morecategories for notes having binding rules with that particular service.In an embodiment, a note is automatically labeled with one or moresuggested categories.

FIG. 6 shows an example user interface 600 for user creation of a noteand user entry of categories and binding rules for the note. In anembodiment, the user interface 600 is accessible via a web browser.However, alternative methods of accessing user interfaces known in theart may also be used. In a preferred embodiment, the user interface 600allows user entry of a first symbol 610 to designate entry of a categoryand a second symbol 620 to designate entry of a binding rule. As shownin FIG. 6, the first symbol 610 is the “ft” sign and categorization isaccomplished by placing a “#” symbol adjacent to a word. Similarly, thesecond symbol 620 is the “@” sign to mark a service for a binding rule.The first 610 and second 620 symbols allow easy entry of categories andbinding rules. As can be appreciated by one or ordinary skill in theart, the present invention is not limited to the use of symbols forcategorization and binding rules.

In a preferred embodiment, entry of the first symbol 610 triggers a listof previously used categories to be displayed to the user. Inparticular, the list can be filtered to only include categories that maybe relevant to the note, such as based on the content of the note or thecontext traits of the note. In an embodiment, users can overridesuggestions and label the note with non-suggested category or a newcategory. Similarly, entry of the second symbol 620 triggers a list ofservices or binding rules to be displayed to the user. In a preferredembodiment, user selection of a category to label the note triggers adisplay of suggested services where the note can be bound. Conversely,user selection of a binding rule triggers a display of suggestedcategories to label the note. The suggestions can be based on previouslystored user selections, i.e. a user history, or any other bases asdescribed herein with respect to the suggestion function.

The user interface 600 also includes a list of categories 630 used inall stored notes. A user can select a category from the list 630 to viewnotes labeled with the selected category. In an embodiment, a search box640 is also available to find previously saved notes. FIG. 6 also showspreviously saved notes 660 and 680. The note 660 includes a photo 665and a weblink 670. The note 680 includes text description of a wine andis labeled 690 with the categories “wine” and “product” by use of thefirst symbol. The user interface 600 also includes a user name 650, suchas the user's email address. In an embodiment, the application serverrequires a user authentication to access the semantic note takingsystem.

Embodiments of the present invention are also directed to userinterfaces corresponding to services to facilitate simple collection ofdata and binding to services. In an embodiment, a user interface isprovided for each of the services where a note can be bound. The userinterface corresponding to a particular service includes one or morebuttons for initiating an action specific to that service. Servicespecific user interfaces are referred to as semantic skins. In apreferred embodiment, a single click of a button on a semantic skininitiates the creation of a note, automatic content generation andaddition to the note, automatic labeling of the note with one or morecategories appropriate to the service, and communication of the note tothe service.

FIG. 7A shows an example of a semantic skin 710 for a social networkservice and FIG. 7B shows an example of a semantic skin 750 for a healthor diet application. The semantic skin 710 includes a button 720 to“poke” a friend of the user in the social network, a button 730 tosubmit the current status of the user, and a button 740 to post a phototo the social network The semantic skin 750 in the example shown by FIG.7B includes a button 760 to enter exercise activity, a button 770 toenter diet information, and a button 780 to measure medical readings.The semantic skins can rely on context traits derived from the devicewhere the data entry is performed. For example, the button 730 ofsemantic skin 710 can utilize the current location of the user asmeasured by the user device and publish that location to the socialnetwork. In another example, a user is operating a device that includesa heart monitor and the button 780 extracts heart rate measurements,attaches the measurements to a note, and communicates the note to ahealth service.

In a preferred embodiment, buttons in user interfaces and semantic skinsare programmable. The buttons serve the purpose of providing simple orsingle click entry of data and minimizes the friction for a user tocollect information and bind the information to services.

FIG. 8, FIG. 9, FIG. 10, and FIG. 11 include details of a functionalembodiment of the present invention. The exemplary embodiment shown inFIG. 8, FIG. 9, FIG. 10, and FIG. 11 are directed to social networkservices, such as Facebook.com and Twitter.com, though its extension toother services can be appreciated by one of ordinary skill in the art.The present invention is not limited to the embodiment of FIG. 8, FIG.9, FIG. 10, and FIG. 11; alternatives to the services and applicationsdescribed in FIG. 8, FIG. 9, FIG. 10, and FIG. 11 can be identified byone of ordinary skill in the art.

FIG. 8 and FIG. 9 show the technologies used by the application serverin an exemplary embodiment of the present invention. The core servicesare shown by FIG. 8 and include the AuthenticationService, NodeService,NodeMetaService, LabelService, SearchService, UndoManager, QueueService,LogService, and PrivacyService. Each of the core services has its owninstance of a BerkeleyDB Java Edition as a backing store. FIG. 9 showsan initial user request 910 that is handled by a web server, e.g.lighttpd. Lighttpd serves up all the static content. If the request isfor dynamic content then it is forwarded to the jetty Java ApplicationServer. Jetty runs inside of the Java Virtual Machine. In the embodimentshown in FIG. 9, the struts2 web framework is used inside of Jetty.Struts2 determines which services are needed to fulfill the request anduses Google Guice to create all the objects needed by the user. Afterall the objects have been created, struts2 fulfills the users requestand generates the page. The jQuery JavaScript library is used to speedup and simplify cross-platform development. Google collections Libraryis used to augment the standard Java collections.

FIG. 10 shows a flow chart related to the AuthenticationService of anexemplary embodiment of the present invention. The AuthenticationServiceallows a user to communicate with any service that requiresauthentication or login information; thereby the semantic note takingsystem can be integrated with any number of third party applications andservices. In the example shown in FIG. 10, a user request to share anote on Facebook is shown. The request is handled by struts2, whichchecks if the user is a valid user of the semantic note taking system.If the request is from a valid user, a request is sent to theFacebookService to verify if the user has valid Facebook credentials. Ifthe user has a valid Facebook account, the request is sent to theQueueService to en-queue the Facebook share request.

FIG. 11 shows a flow chart related to an embodiment of the QueueService.When an event is en-queued in the QueueService's queue, the QueueServicechecks if the note associated with that event has been previously sharedto Facebook. The LogService tracks when a note has been successfullyshared to Facebook. If the note has been previously successfully shared,it is removed from the queue. The QueueService then checks if there wasa problem sharing the note. If the note has had a failure, the shareevent is timed out and put back in the queue to be run after X minutes.The note is then pushed to the user's Facebook account. The note ispushed by the QueueService calling the FacebookService to fulfill theshare request. If the request succeeds, the event is removed from theQueueService's queue and logged to LogService.

EXAMPLES

The following examples demonstrate exemplary applications for theembodiments of the present invention. It is noted that applications forthe present invention are not limited by these examples; otherapplications of the present invention can be identified by one ofordinary skill in the art.

Example 1 Product Review

A user enjoys a product, such as a glass of wine. During a dinnerengagement the user jots down information related to the wine, such asthe brand, vintage, variety, etc., by accessing a user interface to thesemantic note taking system. The note includes a label of “#wine” forfuture reference to the note, Context traits, including the GPScoordinates where the note was created is associated with the note.Using the UPS coordinates and accessing a directory information module,the name of the restaurant is derived and automatically associated withthe note. The note is saved in the database.

At a later date, the user chooses to discuss the wine with friends ofthe user who may be wine enthusiasts. By binding the wine review note toa social network or an email application, the note is communicated toone or more friends of the user. One of the friends writes a briefreview of the wine and attaches that review to the note. After the notehas been shared to friends of the user, the user decides to write a winereview to be posted on a wine website. The user refines the note andbinds it to the wine review website to be published to the world.

Example 2 Retail

FIG. 12A and FIG. 12B show an example of a retail application withbidding from multiple online retailers 1230. In FIG. 12A, user U₁creates a note 1220 on a user device 1210. The note 1220 includes anidentification of a product and binding rules that indicate it is to bebound to retailers R₁, R₂, and R_(N). The note 1220 is communicated tothe retailers R₁, R₂, and R_(N) based on the binding rules. After thenote 1220 is received by the retailers R₁, R₂, and R_(N), the retailerssend offers 1240 to user U₁ for the product described in the note 1220.Alternatively or additionally, the retailers R₁, R₂, and R_(N) can sendadvertisements, coupons, or any information relating to the product tothe user U₁.

Example 3 Health

The present invention can be directed to a dietary, health, or medicalservice. For example, with minimal friction, a user can submit his orher dietary behavior to be tracked by a diet service. In anotherexample, medical information can be collected at a hospital visit ordoctor's appointment and submitted to an electronic medical record (EMR)application.

Example 4 Research

The semantic note taking system of the present invention can also beapplied to a research study. In an embodiment of the present invention,research data is collected in a note. The research data can be collectedfrom the field via a mobile device. The research data is enriched withtime and location context traits. For repetitive data collection, asemantic skin would provide simple one click user entry of data. Forexample, the present invention can facilitate frictionless datagathering from door-to-door surveys. A door-to-door surveyor would neednot enter the address of each participant of the survey as locationcontext traits would be automatically associated with the note. Relevantdata (e.g. resident names and demographics) can be automaticallyaccessed based on the location context traits.

Another aspect of the present invention is directed to semantic notetaking systems that link physical objects to the notes. In anembodiment, the system employs a two-dimensional bar code, such as aQuick Response (QR) code, that can be placed on or near a physicalobject. When imaged, the code links the physical object with one or morenotes, such as by associating a public or private URL with the code. TheQR code is preferably captured with a digital camera on a mobile phone,however, other methods of capturing the QR code can be used. The URLassociated with the code can also be associated with notes from asemantic note taking system as described above, thereby the physicalobject is connected with the note.

The following describes an embodiment of a semantic note taking systemand method with QR codes. First a note is created, enriched, and boundto services as described herein. A QR code is then generated and put onan object in the physical world. The object with the QR code is thenviewed or imaged, preferably with a cell phone camera. As describedabove, the QR code is used to link the object with the note. It isimportant to note that contextual information can also be linked,including the location where the object was imaged, the history of thenote, and other available contextual information. In addition, the notemay be a shared note in an ad hoc network of users, therefore the othermembers of the ad hoc network are also linked with the imaged object.

In one embodiment, a private URL linked with the QR code and a note isavailable for users to hind with services or share with other users. Inanother embodiment, the QR code is linked with an encrypted link forrestricted use, such as for managing private records of an organization.In this embodiment, encryption of the QR code can require an encryptionkey that is only possessed by authorized viewers, wherein the QR code ismeaningless without the encryption key. An authorized viewer can viewthe QR code with his or her phone to generate encoded text, convert theencoded text using the encryption key into a link to a private URL,Encryption of the QR code allows it to be placed in public locationswhile maintaining privacy. For example, inventory located in a publiclocation can be labeled with an encrypted QR code that allows authorizedpersonnel to view the link associated with the QR code, but prevents anyunauthorized views.

In an embodiment, a note is a shared note between one or more users anda QR code is used to link physical objects with the shared note. Theshared note includes a chronology and history of related comments,replies, and links to objects and/or locations. For example, the sharednote includes a chronological list of items, such as user comments andpostings, with the latest item at the top of the list. It is noted thatone or more of the items includes links with objects labeled with a QRcode.

In another embodiment, a shared note is related to a physical objectlabeled with a QR code. The note includes enriched and contextualinformation related to the object and a list of items (e.g. comments andreplies) shared to the community of users having access to the sharednote. Preferably, the items are listed in chronological order.

FIG. 13A and FIG. 13B show a preferred embodiment of the presentinvention with automatic generation of a QR code from a shared note andaccessing the note using the QR code. FIG. 13A shows a shared note 1310with a list of user entries. The shared note 1310 can include aconversation between multiple users. Preferably, the comments andentries in the conversation are listed in chronological order with themost recent entry at the top. A private URL 1320 is associated with thenote. The private URL 1320 provides a link to the entire conversation ofthe shared note 1310 and all of the content of the shared note 1310. Anencryption code 1330 is generated to allow only authorized users fromaccessing the private URL 1320, and thereby the shared note 1310. Amatrix code, such as a QR code 1340, is automatically generated toprovide a link between the note and a physical object.

FIG. 13B shows an exemplary process of updating the shared note 1310using the QR code. First, the QR code 1340 is viewed or imaged by auser, such as by placing the QR code 1340 proximate to a physical objectof interest and imaging the QR code 1340 with a cell phone camera. Ifthe user is an authorized user with the proper encryption key, the usercan then decrypt the encoded text in the QR code 1350 to reveal theprivate URL 1320. The private URL 1320 provides a link to the sharednote 1310 and the user can add an entry into the conversation in theshared note 1310. In the embodiment shown in FIG. 13B, the new userentry is placed at the top of the shared note 1310 as it is the mostrecent entry.

It is important to note that the QR code 1340 allows for a back andforth between one or more physical objects with a shared note 1310. Inother words, the process of FIG. 13A and FIG. 13B can be repeated anynumber of times, thereby physical objects labeled with the QR code 1340become linked with the conversation of the shared note 1310.

The following is a list of example applications for semantic note takingwith QR codes. It is noted that this list is not exhaustive and one ofordinary skill in the art will appreciate alternative applications orchanges to these applications.

-   -   Home or office inventory: QR codes and semantic notes can be        used to inventory items in the home or office. For example, QR        codes can be attached to wine bottles in a wine cellar and        linked to the corresponding wine review notes. FIG. 14 shows an        example of linking a physical object (e.g. wine bottle 1430)        with a note 1420. A QR code 1440 is attached to the wine bottle        1430. The note can be written on a mobile phone 1410 and the QR        code 1440 can be captured with the camera on the same mobile        phone 1410 or a different phone.    -   Health. Diet, and Exercise: QR codes can be attached to        medication to track use and refills. Food labels can rely on QR        codes for a diet log. Exercise equipment can have QR codes to        manage an exercise regimen.    -   Equipment service and repair: By affixing QR codes to equipment        and linking them with notes, a usage and repair history of the        equipment can be stored and monitored. The QR codes can be        encrypted for equipment that is accessible in a public location.    -   Janitorial: QR codes can be used for documenting cleaning and        inspections.    -   Location awareness: By placing QR codes at a particular location        (e.g. a restaurant, place of interest, etc.), a user can track        and remember where they had a certain experience that is        described in a note.

In an additional embodiment, the invention can be used to motivatepositive behavior change by a sense of progress. To achieve this senseof progress, we need to be able to measure and quantify the experiencesthat are on the trajectory toward the behavior change goal. Furthermore,to understand the ups, downs, and gaps, we need to annotate thatprogression with other experience data from the lifestream. Theexperiences that may be the most relevant to the behavior change goalare often not in the same stream as the key progression metric.

There is a discontinuity between behavioral metrics and relatedexperience data. For example, with hypertension the experience data isblood pressure, while the behavioral metrics is the consumption of fatand salt or life experiences and relationships that cause stress. Inobesity, the experience data is weight, while the behavioral metrics isdiet and exercise or life experiences and relationships that enhance orinterfere with positive behavior change. With fitness, the experiencedata is the frequency and duration of exercise, while the behavioralmetrics is the available time and environmental factors or lifeexperiences and relationships that enhance or hinder progress.

Examples of experiences and relationships that enhance or hinderprogress are for example people who nourish and encourage versus peoplewho provide negative feedback, people who engage in similar or contrarybehaviors to the desired behaviors, work and family schedule that appearto leave little time, work and family relationships that cause stressversus relationships that reduce stress, environmental factors andlocations, new priorities and interests that distract from behaviorchange goals, and/or financial factors.

Sources of experience data can come from a variety of areas such associal network environments (e.g. Facebook, Twitter and other bloggingplatforms), financial services (e.g. Mint.com), note services (e.g.Evernote), location check-in apps (e.g. Foursquare), email,photo-sharing services (e.g. iPhoto) or calendars. Social networkingplatforms can be a source of positive and negative reinforcingrelationships through identification of who comments on posts togetherwith sentiment analysis of those comments. They can also reflectinterests and identify source of potential new supporters andencouragers who are not yet connected to the user, Financial servicescan reflect stress and competing priorities for time and attention tobehavior change goals. Note services can identify interests, changingpriorities, as well as work/home life balance and location. Locationcheck-in apps can identify location, interests, and new socialconnections that can support behavioral goals. Email can identifyrelationship patterns and communication frequency, and a sentimentanalysis. Photo sharing services can provide location, priorities,interests, mood, or the like. Calendars provide insight in socialactivity level, work/home life balance, available time, priorities, orthe like.

Behavior-related metrics can be quantified by various sources, forexample, without limitation: fitbit and other pedometers, sensors insmartphones indicating location motion, withings and other electronicscales that can publish weight to the cloud or to smartphones, bloodpressure, blood glucose, respiratory monitoring and other medicaldevices that can publish data to the cloud or to smartphones,thermometers that can indicate illness, exercise loggers such asRunkeeper.com, or any other depending on the type of behavior that onewould like to quantify.

Sources of quantified behavioral data can be Catch.com mobile Apps thatallow for user-defined shortcuts for logging data. It further allows forthe integration with third party applications that generate structureddata. It also allows for semantic analysis of free-text, imaging orvoice information for quantified parameters. For example, frequencytags, frequency and timing of words and phrases, sentiment analysis,stress prediction from voice notes. Another example is same semanticanalysis that also run against third party data from other linkedservices. Apps like Catch.com further allow for capturing phone sensordata as context and can harvest data from other devices in the vicinitythrough near field communications (NEC).

FIG. 15 shows an example of a system outline of a structure withexperience data sources annotating behavioral progress application,which the following items:

1. Catch lifestream application.2. Catch mobile App communication with Catch cloud via Catch API.3. Catch lifestream data and annotations (metadata attached toexperiences captured in Catch),4. Communication with third party services via API.5. Facebook and other social networks.6. Evernote and other note services.7. Gmail and other email services.8. Fitbit, Withings, and other devices that publish to cloud serviceswith open APIs,9. Other devices and services.10. Third party cloud-based services communication with their own datacapture devices.11. Facebook and Twitter, web or mobile client.12. Evernote mobile client or desktop App.13. Gmail web or mobile client.14. Fitbit pedometer, Withings scale or other sensor-based devices.15. Other devices or interfaces that send data to cloud-based services.

FIG. 16A, FIG. 16B, and FIG. 16C show examples of Catch.com functionsand annotated lifestream. In FIG. 16A, FIG. 16B, and FIG. 16C, we show:

1a: Shortcuts that capture more structured information and add it to theCatch note stream as an annotation. These shortcuts also can come fromtemplates designed by third parties.1b: In-phone data sources that are attached to each note, including thenotes generated by the shortcuts in 1a as well as any other informationcaptured on the phone that is added to theCatch note stream.1c: Annotated feedback

-   -   For the behavioral parameters that the user decides to track,        the app can graph and chart the data.    -   Ups, downs, and gaps are enriched by coincident data from the        stream.    -   Coincident data can be from Catch notes or any third party        service with a linked account to Catch.        1d Example: peak is coincident with a note in Evernote, and        would be flagged as such.

Clicking on the 1d note would bring the user to the source data.

1e Example: decline or gap could be informed by an email message orFacebook post that was coincident.

In summary, this embodiment provides a behavioral modification systemwhereby a mobile phone is able to run applications that log userbehavior and phone sensor data. The system has the ability to linkaccounts to third party information capture sources with time-stampeddata. It further has the ability to define quantified metrics and tochart progress against such metrics. It also has the ability to chartprogress metrics with flags for coincident information with ups, downsand gaps in progress metric. Yet other abilities are, the ability toidentify directional changes, ups, downs, gaps in progress metric and(e.g. ability to identify associated friend or social communications),ability to identify associated locations, ability to identify associatedinterests, priorities, and other information that is based on semanticand sentiment analysis of content from third party services, locationdata, tags and metadata, and device measurements.

In other embodiments, we described the idea of being prompted by theapplication for data entry based on location and other context. We alsodescribed using context such as location and orientation of the phone tohelp determine the content of data captured from imaging. In yet anotherembodiment, the invention expands the idea to include templates orsoftware modules that are selected and run within Catch based oncontext.

Catch is a personal content management system that is easy to adapt anduse for a wide variety of information capture scenarios. People useCatch for everything from expense reports and meeting notes, torecording images and notes about family and home life, to recordingexercise and diet activities. Information can be viewed as separateactivity streams based on tags, location, people, and other metadata.This embodiment, uses context more richly to tailor the Catchapplication to differentiate between home and work contexts, or betweenfamily life or self-improvement contexts.

FIG. 17A, FIG. 17B, FIG. 17C, and FIG. 17D show examples ofcontext-aware modules for tracking, insights and coaching feedback.Referring to FIG. 17A, FIG. 17B, FIG. 17C, and FIG. 17D, templates canbe associated with any combination of contextual data gathered on thephone. Examples include:

Location: Home versus work, gym versus restaurant.

Motion: in a vehicle versus stationary, patterns that indicate walkingor jogging.

Orientation: Pointing up or down, landscape versus portrait mode.

Time of day: Morning activities versus evening.

Past patterns and user curation of past associations: Looking at notehistory and associations, allowing user to correct associations andremembering those results.

Objects in vicinity: Near field communications: near a weight scale ortreadmill, near a cash register, other objects, at a business orretailer, anything identifiable via NPC.

Day of week, month, season: Weekend versus weekday, holidays, seasonalfactors, for example season in combination with location, motion.

Communication behaviors: Phone, email, calendaring, existence or absenceof appointments.

Referring to FIG. 17B, templates are modules that a user can load intoCatch to add. Examples are shortcuts for capturing data with additionalstructure. Data can be stored as human readable notes annotated by datafields that are more easily machine readable. Another example isanalytic rules defining which metrics should be charted with which othermetrics, the scales and timeframes of those charts, and anypre-processing of data. Yet another example is content feeds forfeedback and coaching, which could include rule-based content used toannotate a chart, or feeds of content from an expert source filteredbased on user context and history. Templates can be user configurable,can be associated with a specific activity (such as scuba diving,fishing and other outdoor activities, hobbies, travel, cooking, diet,fitness'), or can be associated with a partner or brand who develops anddefines the shortcuts, analysis rules, and content feeds.

Referring to FIG. 17C and the quantitative analysis, each shortcutcreates notes and annotations or metadata attached to notes. Thesemetrics can be charted on a time scale. Rules for how metrics arecharted include:

X and Y scales, as in which time period or range should the variable beviewed.

Chart type, such as bar, line, or scatter plot.

Other metrics that should be viewed on same chart at the same time.

Pre-processing of data, for example any formula that runs on the rawdata or data in association with context to produce the result that willbe charted.

Referring to FIG. 17D, feedback and coaching, content from the modulecan be linked directly to the chart as a flag that can be clicked on totake the user to a more complete explanation. Content associated withgraphs can be tagged to ups, downs, gaps, or any if-then scenario usingdata from the metrics or context as input. Content also can be displayedas a feed from the module sponsor or author. Content feeds would befiltered based on user data including profile, context, and trends inmetrics.

Data sharing and privacy properties can be associated with a template.Examples of data captured with a template are:

for scuba divers could be shared with a research community aggregatingdata about the health of the coral reef.

associated with a health condition could be aggregated and shared withresearchers studying disease progression.

developed by a pharmaceutical template could be shared with the sponsor

for people losing weight or joining a fitness program could be sharedwith friends on Facebook or Twitter.

Each module can have a terms of service addendum that explains andqualifies the data sharing rules.

To provide positive psychology (e.g. FIG. 18A, FIG. 18B, and FIG. 18C),as a productivity tool sponsored by an employer, it could be the defaultmodule during working hours. Anonymous data is shared with research teamstudying the relationship between outcomes and adherence. In an example,Health tracker, as shown in FIG. 19, could be the set as a defaultmodule during mealtimes or when location is related to a previouspattern tagged as exercise. Data from the module could be automaticallyshared with a specified friend list. Data could be aggregated for agroup contest. For example, a group of friends or employees challengingeach other to maintain a health regimen

In an example of an expense tracker, as shown in FIG. 20A, FIG. 20B,FIG. 20C, and FIG. 20D, if the context is a restaurant during businesshours use this template by default, but not on weekends and holidays. Inthe Expense tracker, shortcuts could include place or business whereexpense occurred, people who were present, and business purpose: all thedata needed for expense reports for an employer. Data could beautomatically shared with the company accountant. Analysis rules couldinclude formatting and totaling expense data by category.

In summary of the embodiment, we provide a personal information capturenote-taking system on a mobile device, including a loadable module withshortcuts and rules for processing captured information. Content can becaptured from the loadable module that categories data input forsubsequent analysis. Data can be displayed in a chart based on rules inthe module. Additional content can be associated to the data in thechart based on filters or rules. The loadable module can be selected anddisplayed based on context data collected in the mobile device. Sharingrules can be associated with a module. Data can be annotated in thenotes database with quantitative data in addition to unstructured datacaptured in notes. Annotations are machine readable fields that areattached to a note as metadata. Context can be captured in associationwith notes and using this context to determine which of a plurality ofloadable modules should process the data.

What is claimed is:
 1. A method for collecting information and enrichingsaid information, said method comprising: (a) providing a note takingfunction for allowing a user to create a note on a user device, whereinsaid created note is stored; (b) providing an identification function tocapture user's interests and needs from a plurality of sources ofexperience data to which said user is linked, wherein said sources ofexperience data originate from one or more social networks: (c)providing a template function for defining one or more templates fororganizing data capture and visualizing said organized data and whereinsaid templates are stored; and (d) providing a pre-selection orsuggestion function for selecting one or more of said stored templatesbased on said identified user interests and needs and based on saidcreated note.
 2. The method as set forth in claim 1, wherein said notecomprises text, audio media, visual media, audio-visual media, recordeddata, a web link, a pointer to an information source, or any combinationthereof.
 3. The method as set forth in claim 1, wherein said user devicecomprises a mobile device, and wherein said note is communicated over awireless network.
 4. The method as set forth in claim 1, wherein saidsources of experience data are further one or more financial services,one or more note-taking services, one or more location check-in apps,one or more email programs, one or more photos-haring services, one ormore calendars, or any combination thereof.
 5. The method as set forthin claim 1, further comprising providing a feedback function forproviding an enriched note to said user, wherein said feedback functionperforms based on said pre-selected template a quantitative analysis anda visualization of said quantitative analysis to said user of saidenriched note.
 6. A system for collection information and enriching saidinformation, said system comprising: an application server, wherein aplurality of users are communicatively connected to said applicationserver to create one or more notes, wherein a plurality of services arecommunicatively connected to said application server, and wherein saidnotes can be bound to one or more of said services; and a database forstoring said notes created by said users; wherein said applicationserver operates a plurality of functions, said functions comprising: (a)a note taking function to allow a user to create a note on a userdevice, wherein said created note is stored: (b) an identificationfunction to capture user's interests and needs from a plurality ofsources of experience data to which said user is linked, wherein saidsources of experience data originate from one or more social networks;(c) a template function for defining one or more templates fororganizing data capture and visualizing said organized data and whereinsaid templates are stored; and (d) a pre-selection or suggestionfunction for selecting one or more of said stored templates based onsaid identified user interests and needs and based on said created note.7. The system as set forth in claim 6, wherein said note comprises text,audio media, visual media, audio-visual media, recorded data, a weblink, a pointer to an information source, or any combination thereof. 8.The system as set forth in claim 6, wherein said user device comprises amobile device, and wherein said note is communicated over a wirelessnetwork.
 9. The system as set forth in claim 6, wherein said sources ofexperience data are further one or more financial services, one or morenote-taking services, one or more location check-in apps, one or moreemail programs, one or more photo-sharing services, one or morecalendars, or any combination thereof.
 10. The system as set forth inclaim 6, further comprising a feedback function for providing anenriched note to said user, wherein said feedback function performsbased on said pre-selected template a quantitative analysis and avisualization of said quantitative analysis to said user of saidenriched note.